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Introduction 

• Improving visualization of your data helps in 

• Debugging 

• Verification 

• Understanding 

• Challenge your workflows and tools 
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Simple example 
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Nice properties for a debug tool 

• Minimize impact on client 

• Low memory requirement 

• Low processing power requirement 

• Lean API for minimal debug code 

=> Separate process, communicates over network 

• Clutter-free UI 

• Shouldn't need user's manual for the tool 

• Helps to keep visualization simple as well 
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Part I - Visualizing Runtime Flow 

• What are the main components 

• Who manages the lifetime 

• What is the lifetime 

• What are the dependencies 
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Sequencer 

• Hierarchy to show structure 

• Timeline and tracks to show history 
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Hierarchical Timeline View 
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Part II - Record and Playback Data 

• Simple and data agnostic 

• Register binary feed and callback 

• Add arbitrary data { ID | Time | Byte[*] } 

• Scrub timeline to send back to feed 
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Use in Killzone Shadowfall 

• MP Bot AI debugging and validation 

• Gameplay animation debugging 

• Player 
. NPCs 

• Took ~1 week to integrate and hook-up 
debug calls 
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Part III - Visualizing Algorithms 

• How to visualize algorithms 

• Not just the end result but step-by-step 

• No access to Tenderer 

• Long turn-around time to use in-game 
rendering 

• Also, alternative viewport 
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I've found out that... 


• Visualizing data is not trivial 

• Iterate but keep it simple 

• Time is of the essence 

• Collapse into single image 

• Series of snapshots 
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Behind the scenes 

• ReView communicates using RPC over TCP/IP 

• Major contributor to extensibility! 

• C# for building the tool 
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Quick look at the code 


Feed 0 Connect ("localhost", 5000); 

track_id = Feed AddTrack(parent_id, "Name"); 
item_id = Feed.AddItem(track_id, time, "Name"); 

Feed AddLog(item_id, time, flags, "Log entry"); 

box_id = Feed . AddBox(time, Inf, Matrix. Identity, center, size. Color Green) 

Feed RemovePrimitive(box_id, later_time); 

id = Feed .AddMesh(time, Inf, Matrix. Identity, center, flatShaded : true); 
Feed.AddTriangle(id, time, pointA, pointB, pointC, Color GreenAlpha); 



ARTIFICIAL INTELLIGENCE SUMMIT 


GAME DEVELOPERS CONFERENCE" 2014 MARCH 17-21, 2014 GDCQNF.COM 


Takeaway 

Don't guess what happened... 
...know what happened! 
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That's All! 


Follow @MikaVehkala 

Review can be found at www.reviewtool.net 
Special thanks to Maurizio De Pascale 
Suggested reading; 

Edward Tufte, The Visual Display of Quantitative Information 
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Introduction 


• Our version of a cheap, but powerful 
tool for historical debugging 

• Tools are always worth the time, but 
it's never too late 

• Can be built at very low cost 

• I should've done it sooner 







("Preconditions Evaluated I & I Von umHou i Grapple Sequence preconditions: Done 
[Node Evaluated j BTVenomHound Grapple evaluated: Failed 

iNode Evaluated ] BTVenoniH .1- ■ Grapple Sequence evaluated: Failed 

[Node Evaluated 1 B 1 venomHound Attack Orient evaluated: Failed 

FPf o<. orditions Evaluated j B ! Ven omHou u Atta'ck preconditions: Failed IsTargetPoisoned TargetType 

1 Bf ven omHound Attaak "evaluated : Failed 

[Pi econditions Evaluated 1 B i Ven omHo 1 1 Attack preconditions: Failed isTargetShielded 

[Node Evaluated ] B 1 veriomHouud Attack evaluated ii Failed 

[ Pi ec onditions Evaluated j b l Von omHou n d Backoff preconditions: Failed TargetDistance 

[Node Evaluated 1 B f VenomHOUnd Backoff evaluated: Failed 

[Node Evaluated Area Check evaluated: Done 

" Node Evaluated ] b iv e e ,-niHou Dist ance evaluated: Done 

[Node Evaluated j BTVe-iumHo 1 < TargetS^aluated'T^Done 

[Node Evaluated ] BTVenomHoun Root evaluated: Done 

[Locomotion 1 Position: ( 356.27, 678.48, 240.97 ) Velocity: ( -6.95,. -B.64, - 0.01 ) 


BTVenomHound: 5 steps 


> Root: Done 

> Target: Done 

> Distance: Done 

> Area Check: Done 

> Chase: Executing 


[Blackboard Changed] - 2 events 


player NoName [30583] 
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Background on Evolve and TRS 

• Online cooperative/competitive first/third-person 
shooter 

• Always plenty of AI, even in full online games 

• AI agents also must play all roles in liu of human 
players 

• Rapid development; need to leverage lots of playtest 
data 
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Bare Bones Requirements 


• Get it up and running in a man-week O” 

• Took almost as long to prepare this presentation © 

• Rapidly and safely add data; vis comes second 

• A dedicated server recording should feel like a 
local session 

• Runs on server, so minimum CPU overhead 
during recording 
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Stupid-Simple Data Stream 


• Self-contained events and metadata in a contiguous 
memory stream 

• Metadata typically very small, and easily quantized 

• Store frame markers to establish timeline 

• If the stream is nearly full, we purge old contents 

• "Version 2.0" would handle this more intelligently 

• The data's all there - reconstruct and render later on a 
visual client 





Writing the Stream 


Interpreting the Stream 




New Render Frame 


Timeline scrubs between render frames 
We always know what happened in the 
past, relative to T 

Turn small atomic events into useful data 
Higher granularity than this example 
(details later) 
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Versioning 

• We simply distinguish between 
last-readable version - pretty s 

• Each event type's serialization 
multiple versions 


SERIALIZE () 

{ 

VEC3_RANGE (m_pointStart , 4000 . f ) ; 
VEC3_RANGE (m_pointEnd, 4000 . f ) ; 

if (version >= 3) 

{ 

FIELD (m_f lightType) ; 

} 


} 





Periodically strip old version support, just so the code is 
tiny 
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Game Data Compatibility 

• We always know the originating build's stamp; sync to 
data as necessary to reference large data 

• When possible, events store inputs, and re-execute during 
timeline scrubbing 

• Determinism is important, but only needed in a small subset of 
systems 

• Some events just serialize results if they're tiny 
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Minimizing Metadata 


Referencing Static Data 


NodeEvalFailed 


Metadata 

Node ID 

Failed Precon ID 


Minimal, just relates directly to static BT data 
Sync to older game data as necessary 


Re-Query With Stored Input 


TPSQuery 


Metadata 

Query ID 
Context 


PathFind 

P 

FlightPath 

1 

Etc... 



For a tactical query, we need to see all 
candidates and their scores (tons!) 

Way too much to store, so we store the context 
used to conduct the query 

Just re-execute tactical query; metadata as input 
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Playback & Scrubbing 



• Timeline shows a range of time with color-coded markers 

• Linearly process entire stream up to the displayed frame 

• Use gamepad to scrub back and forth, detach camera, 
select different agents 

• Aggregating larger context under the hood for a complete 
picture 

• Anything traditional debug displays can show... but with 
history 








[Locomotion 


1 Position: ( 490. Q6, 723.89, 250.00 ) Velocity: ( 


[Blackboard Changed] - 1 events 

AngleToTarget => 47.3047 



BTMerc Med icP rimary: 4 steps 

> Root : Done 

> Medic: Done 

> Positioning: Done 

> Follow Position: Executing 

BTMerc MedicSecondary: 2 steps 

> Root: Done 

> Idle: Executing 
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Version 2.0 impToroli 

• Obvious next step is to visualize in external app 

• Though, something to be said about being in-game 

• Stream over the network, "infinite" history 

• Or write events to a DB, such as a free NoSQL 
key/ value store 

• Visualize on the web or anywhere else 

• Better visualization, animation/position rewind 

• In the works, bit-by-bit as necessary 
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Conclusions 

• So much data: from any bug report, we 
have recent history for all active agents 

• We see everything that's happened on a 
remote dedicated server 

• Engineers new to the team were able to jump in and track 
down tricky bugs in a fraction of the time... says Troy 

• We observed and fixed bugs we weren't even looking for 

• Replaced all the disjointed visualization junk we had 
before 


I will have my 

ciake, 

and eat it too 


* almost 
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Conclusions 

If you have a need for historical debugging 
and have no resources to spare, try 

something like this. 


You won't regret it. 




